Acquisition and synchronization of multimodal articulatory data
نویسندگان
چکیده
This paper describes a setup to synchronize data used to track speech articulators during speech production. Our method couples together an ultrasound, an electromagnetic and an audio system to record speech sequences. The coupling requires a precise temporal synchronization, to know exactly the delay between the recording start of each modality, and to know the sampling rate of each modality. A complete setup and methods for automatically synchronizing data are described. The aim is to get a fast, low-cost and easily reproducible acquisition system in order to temporally align data.
منابع مشابه
Articulatory Synthesis of Speech and Singing: State of the Art and Suggestions for Future Research
Articulatory synthesis of speech and singing aims for modeling the production process of speech and singing as human-like or natural as possible. The state of the art is described for all modules of articulatory synthesis systems, i.e. vocal tract models, acoustic models, glottis models, noise source models, and control models generating articulator movements and phonatory control information. ...
متن کاملEvaluation of the uncertainty of multimodal articulatory data
Our purpose is to present the last improvements brought to our acquisition system of multimodal articulatory data and reports on a complete evaluation of the uncertainty attached to the data it provides. In previous works [1, 2, 3], we presented our system to acquire multimodal articulatory data. The data are both dynamic: 2D Ultrasound (US) to get midsagittal images of the tongue at 66 Hz, ele...
متن کاملMultimodal Fusion of Electromagnetic, Ultrasound and MRI Data for Building an Articulatory Model
Data fusion from multiple sensors is of significant interest to the speech research community, as it can potentially provide a better picture of speech production through the use of complementary sensor modalities. This paper deals with the practical aspects of this problem, such as acquisition and processing of the dynamic ultrasound (US) and electromagnetic (EM) data of the tongue during spee...
متن کاملPhonological Awareness Impact on Articulatory Accuracy of the Spanish Liquid [r] in Japanese FL Learners of Spanish
Foreign language learners tend to avoid phonological difficulties and simply transfer sounds whether from their L1 or any pre-existing L2. Phonological awareness (PA) gives students an active role in understanding their own potential in improving pronunciation through several methods. However, such methods are likely to be restricted to only passive learning methods, such as repetition, reading...
متن کاملSynchronization of Multimodal Recordings for Musical Performance Research
The past decade has seen an increase of low-cost technology for sensor data acquisition, which has been utilized for the expanding field of research in gesture measurement for music performance. Unfortunately, these devices are still far from being compatible with the audiovisual recording platforms which have been used to record synchronized streams of data. In this paper, we describe a practi...
متن کامل